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Abstract — We illustrate the utility of the recently developed 
loop calculus [1], [2] for improving the Belief Propagation (BP) 
algorithm. If the algorithm that minimizes the Bethe free energy 
fails we modify the free energy by accounting for a critical loop 
in a graphical representation of the code. The log-likelihood 
specific critical loop is found by means of the loop calculus. 
The general method is tested using an example of the Linear 
Programming (LP) decoding, that can be viewed as a special 
limit of the BP decoding. Considering the (155,64,20) code 
that performs over Additive- White-Gaussian channel we show 
that the loop calculus improves the LP decoding and corrects 
all previously found dangerous configurations of log-likelihoods 
related to pseudo-codewords with low effective distance, thus 
reducing the code's error-floor. 

Belief Propagation (BP) constitutes an efficient approxi- 
mation, as well as an algorithm, that applies to many infer- 
ence problems in statistical physics [3], [4], [5], information 
theory [6], [7], [8], [9], and computer science [10]. All 
these problems can be stated in terms of computation of 
marginal probabilities on a factor graph. If the underlying 
graph structure contains no loops, i.e. it is a tree, the BP is 
exact, being only an approximation in the case of a general 
graph. The BP approximation can be restated in terms of a 
variational principle [11], [12], [13], where the BP equations 
describe a minimum of the so-called Bethe free energy, and 
the standard BP algorithm [6], [7] means solving the BP 
equations iteratively. 

In coding theory BP plays a special role as the decoding 
of choice for Low Density Parity Check (LDPC) Codes 
introduced by R. Gallager [6]. These codes, described in 
terms of sparse (Tanner) graphs, are among the best per- 
forming codes known to date. Actually, these codes perform 
so well exactly due to the high-quality performance of the 
computationally efficient BP decoding scheme [6], [8], [9]. 
In the water-fall domain, i.e. at low and moderate Signal- 
to-Noise Ratios (SNR), the Frame-Error-Rate (FER) or Bit- 
Error-Rate (BER) of an LDPC code decoded using BP comes 
close to the optimal yet inefficient Maximum-Likelihood and 
Maximum-a-Posteriori decodings. However, in the low noise 
regime BP decoding clearly fails to approximate ML in 
practical (finite size) LDPC codes, thus causing the error- 
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floor [14], [15], [16]. It is now well understood that the BP 
decoding failure in the error-floor domain is due to existence 
of pseudo-codewords [17], [18], [19], [20], [21], [22] and 
related instantons [23], [24], defined as dangerous noise con- 
figurations causing the failures. Removing the obstacles and 
thus improving the BP decoding while keeping a reasonable 
computational complexity is on a great demand for high- 
performance applications, e.g. optical communications and 
data storage, where error-floor is a serious handicap. 

Some general purpose BP-improvement strategies were 
already discussed in the literature [13], [25], [26]. Survey 
Propagation (SP) has been suggested in the context of 
combinatorial optimization [25]. SP, based on the so-called 
replica approach that originates from spin glass theory [27], 
applies successfully to highly degenerate problems where the 
standard BP would be trapped in a local unrepresentative 
minimum. Generalized Belief Propagation (GBP) of [13] 
constitutes another generalization of BP. It extends the cluster 
variational approximation of statistical physics [11], [12] to 
problems in information and computer sciences. GBP has 
been shown to perform well for the problems with many 
short loops, like Inter-Symbol Interference on a regular two- 
dimensional lattice [28], where transition to a coarse-grained 
cluster offers an improvement over the standard BP. The 
only yet serious drawback of the method is the expense of 
an overhead that scales exponentially with the cluster size. 
An efficient alternative to GBP, that is claimed successful in 
describing some random graph and random spin models on 
lattices, has been recently discussed in [26]. The method is 
based on closing the system of cavity equations and suggests 
a way to account for correlations (many loops) on all scales. 

In spite of their potential in dealing with highly degenerate 
problems where the bare BP approach fails, the set of 
the aforementioned methods/algorithms (maybe except for 
a possible extension of [26]) do not seem appropriate in 
dealing with the error-floor problem. Indeed, one expects 
that any actual code (as opposed to ensemble of codes) 
has a discrete set of well-defined and relatively simple 
troublemakers (pseudo-codewords) localized on a subgraph. 
Therefore, for each (rare) failure of the standard BP one 
needs to identify and correct for a relatively long correlated 
configuration that is localized however on a small portion of 
the total number of bits. 

This paper suggests an efficient approach that generalizes 
the BP algorithm and which is capable of reducing the 
undesirable error floor effect. Our method is based on the 
recently developed analytical tool, called loop calculus [1], 
[2], that represents the partition function (and, therefore, 



the marginal probabilities) in terms of a finite series where 
each term is associated with a generalized loop on the graph 
and the zero order contribution corresponds to the bare BP 
approximation. We conjecture that for an instanton noise 
configuration that causes a BP failure there is always a 
relatively simple loop correction to the bare BP (in the 
terminology of the loop calculus) that provides an equal or 
comparable contribution to the partition function. We further 
suggest an improved decoding scheme based on finding this 
critical loop correction (in the case of the bare BP failure) 
followed by correcting the error. This is achieved by a 
proper modification of the Bethe free energy and the BP 
equations. These ideas are verified using an example of the 
Tanner (155, 64, 20) code [29] performing over the Additive- 
White-Gaussian-Noise (AWGN) channel and decoded with 
the Linear Programming (LP) decoding [30]. We build this 
test on analysis of the set of pseudo-codewords recently 
found for this code by the LP-based pseudo-codeword search 
algorithm [31]. We introduce the LP-erasure decoding which 
is equivalent to the standard LP decoding with full or partial 
erasure of information at the bits along the critical loop. 
We demonstrate that the LP-erasure algorithm corrects errors 
associated with all previously found pseudo-codewords of the 
Tanner code (~ 200 of them) completely closing the error- 
floor gap between the lowest LP-instanton, with the effective 
distance w 16.4037, and the Hamming distance 20 of the 
code. 

The manuscript is organized as follows. An extended 
introductory Section H] consists of four Subsections. Sub- 
section II-AI introduces the notations and briefly overviews 
decoding of a binary linear code. Subsection II-BI describes 
the loop calculus of [1], [2] also complementing it by some 
new variational interpretation that was not discussed in the 
original papers. Subsection II-CI describes the calculation of 
a-posteriori log-likelihoods (magnetizations) within the loop 
calculus. Subsection II-DI establishes a connection of the 
Bethe free energy approach to the loop calculus and LP 
decoding. Section |ll] unveils an underlying loop structure, 
e.g. emergence of certain critical loops, for the family of 
instantons that appear in the (155,64,20) code performing 
over the AWGN channel and decoded by LP [31]. The 
effective free energy approach, suggesting modification of 
the BP gauges to account for the critical loop, is introduced 
in Section ||ll] An improved LP decoding, called LP-erasure, 
is presented in Section Hvl Numerical test of the LP-erasure 
algorithm using an example of the (155,64,20) code is 
discussed in Subsection IIV-AI where we also demonstrate 
that all previously found bare LP instantons (most damaging 
noise configurations) are actually corrected by the LP-erasure 
procedure thus completely reducing the error-floor observed 
for the standard LP decoding. The final Section IV1 contains 
conclusions and discussions. 

I. Introduction 

A. Decoding in terms of Statistical Inference 

A message word that consists of K bits is encoded to 
an A^-bit long codeword, N > K. In the binary linear case 



the code can be conveniently represented by M > N — K 
constraints, usually referred to as parity checks or simply 
checks. Formally, n — (wi, . . . , ttn) with -Ki = ±1, is one 
of the 2 K codewords if and only if Yl iea % i = 1 f° r a ^ 
checks a — 1 , . . . , M, where i g a if the bit i contributes 
the check a. The relation between bits and checks (we use 
i 6 a and a 3 i interchangeably) is often described in 
terms of an M x N parity-check matrix H that consists of 
ones and zeros: H a i = 1 if i £ a and H a i = otherwise. 
A bipartite graph representation of H, with bits marked as 
circles, checks marked as squares and edges corresponding to 
the corresponding nonzero elements of H, is usually called 
the Tanner graph associated with the code. For an LDPC code 
H is sparse, i.e. most of the entries are zeros. Transmitted 
through a noisy channel, a codeword gets corrupted due 
to the channel noise, so that the channel output at the 
receiver is x ^ n. Even though an information about the 
original codeword is lost at the receiver, one still possesses 
the full probabilistic information about the channel, i.e. the 
conditional probability P(x\cr) for a codeword er to be a 
pre-image for the output word x is known. In the case of 
independent noise samples the full conditional probability 
can be decomposed into a product, P(x\cr) = YiiP( x i\ a i)- 
The channel output at a bit can be conveniently characterized 
by the so-called log-likelihood hi = log(p(xi \ + l)/p(xi \ — 
l))/2s 2 measured in the units of the Signal-to-Noise-Ratio 
(SNR), normally defined as 2s 2 . For a common model of the 
Additive White Gaussian Noise (AWGN) channel , p{x\a) — 
exp(— s 2 (x — a) 2 / 2) / 2tt / s 2 , hi — Xi. The decoding goal 
is to infer the original message from the received output x. 
ML decoding (that generally requires an exponentially large 
number 2 K of steps) corresponds to finding a maximizing 
the following weight (probability distribution) function 

W{&) = Z- 1 \\ S 1 11 a l: +1 ) cxp ( J2 ) . (!) 

a \iea / \ i / 

where the normalization factor Z that enforces the 
Y^ia- W(cr) = 1 condition is called the partition function 
in the statistical physics literature. Maximum-A-Posteriori 
(MAP) decoding boils down to finding a-posteriori log- 
likelihood (magnetization) at a bit defined according to 

m = J2<rW(<r), (2) 

er 

followed by taking an absolute value of the result bit-wise. 

Even though bits and checks play an essentially different 
role in the LDPC decode, it actually turns out to be formally 
convenient to consider them on equal footing thus putting 
the relevant inference problem in a more general context 
of graphical models [32], [33], [34] where binary variables 
are shifted from bits/vertexes to edges of the corresponding 
Tanner graph. 

A general vertex model, also called normal factor graph 
model [33], is determined by the weight (probability distri- 
bution) function W, which along with the partition function 



Z can be represented in the following general form 
W(a) = Z- 1 [J f a (cT a ), Z = J2 II /-( <T «)' < 3 > 

aeX cr aeX 

where X defines a graph consisting of vertexes and edges; a 
denotes a node (vertex) in the model; an elementary spin 
resides at the edge connecting two neighboring vertexes, 
a a t = ±1, for b G a and a E b; <r a stands for the vector 
built of all (Tat, with b G a; cr is a particular configuration of 
spins on all the edges. With this notation one assumes that 

Cat = CTfca- 

The problem of LDPC decoding Q represents a particular 
case of the general vertex model with bits and checks 
combined in one family of vertexes {a} — {i} U {a}, the 
Tanner graph X and the factor functions defined according 
to 

= | exp(/ij<7,), a la = (Tip =<Ji Va,/3 3 i ^ 



otherwise; 



m<x q )- I o, rw* = -i 



(5) 



Our strategy will be to keep the general vertex model 
notations whenever possible. The transition in the general 
formulas to our focus case of the LDPC decoding will always 
be simple and straightforward according to Eqs. J4I5 i . 

B. Loop Calculus [1 ], [2 ] 

Consider a general vertex model and relax the con- 
dition (Jai, = a}, a , thus treating er ao and <7f, a as independent 
binary variables. We represent the partition function in the 
form: 

1 + ObcO~cb 



(6) 



be 



Note that for this representations the vectors <r a become 
independent variables. Also in the product over edges, (6c), 
we assume that each edge contributes only once. We further 
introduce a parameter vector r/ with the set of independent 
components r\ a b- Making use of the algebraic relation 

cosh(?7 + x)(l + na) 



= 1 



(7) 



(cosh 77 + a sinh 77) (cosh \ + t sinh \) 
(tanh(?7 + x) — cr) (tanh(?y + x) ~ 7r ) cosh 2 (77 + \)- 

we arrive at the following representation for the partition 
function that is ready for a subsequent loop decomposition 

z = (n 2 c ° sh + ^) ) Yj n p * n ^ (8) 

\bc / <r a be 

P a (<Ta) = fa(<r a ) JJ (cosh i] ab + o ab sinhr/ ab ) ; (9) 



bea 



V bc (o-fcc, cr c b) = 1 + (tanh(7y fcc + ?7 cfc ) - cr hc ) 
* (tanh(77 hc + T] cb ) - (T c b) cosh 2 (r/bc + r) cb ). 



(10) 



Eqs. J8I91 1 Oi are generally valid for any choice of the 77 
fields (gauge choice). However, in the rest of this paragraph 
we will be discussing a particular choice of the gauge fields, 
r/( bp \ special because of its relation to Belief Propagation. 



The desired decomposition is obtained by expanding the 
y-terms followed by a local computation. The parameters 
(gauges) T] are chosen using the criterion that subgraphs with 
at lest one loose end do not contribute to the decomposition. 
This can be achieved if the parameters satisfy the following 
system of equations: 

(tanh(^ P) + V { b b a P) ) - °ab) Pa{cr a ) = 0. (11) 

It is also straightforward to check that the gauge-fixing 
condition Eq. dl U corresponds to an extremum of Zq, 



SZ 



$Vab 



= 0, 



where 



Z 



JJ 2 cosh {r] bc + Vcb) j EII Pa (' TQ ) 

\ be ) cr a 



(12) 



, (13) 



is the bare part of the partition function Z derived from 
Eq. l|8} with the product of the vertex y-terms replaced by 
unity. 

Eqs. constitute the BP system of equations, repre- 
sented in terms of parameters ri^ bp \ Calculated within BP the 
probability of finding the whole family of edges connected 
to a node a in the state <r a is 



b (bp)( \ _ Pa{°a) 

a [ a) 22„PM 



(14) 



A typical sum, entering a diagram contribution for a gener- 
alized loop C, is expressed in terms of the corresponding 
irreducible correlation functions of the spin variables com- 
puted within BP: 



u (bp) = 

Mo 



e^k) n (** 



-Kb ) 



(15) 



b£a,C 



where m 



(bp) 

ab 



is the magnetization (a-posteriori log- 



likelihood) at the edge (ab) calculated within BP 



(16) 



Making use of Eqs. (I8I15I16> one derives the following final 
expression for the partition function 



Z (\ + 5>(C)^ , 



r(C) 



II Mo 



(bp) 



n (i-rf) 2 ) 

(ab)£C 



(17) 



where Zq is taken at rf hp ^ and the summation in Eq. runs 
over all allowed C (marked) paths in the graph associated 
with the model; (ab) marks the edge on the graph that 
connects nodes a and b. A marked path is allowed to branch 
at any node/vertex, however it cannot terminate at a node. 



We refer to such a structure as a loop (it is actually some 
kind of a generalized loop since branching is allowed; we 
use the shorter name for convenience). An example is given 
in Fig. Q 

In the LDPC case ( 1415 \ the loop series expressions ( I17> 
assumes the following form 



Z LDPC = Z Q ^l + ^ r (C)^ 



(18) 



a3i 



r(C)= 1] M^VT. »=E 1 ' 

(1 - mf p) )« i_1 + + mf^)^" 1 



(6 P ) 



2(1 - (mf) 2 )*" 1 



where fcf'VO and &i" p) K) 

are the beliefs defined on bits 
and checks respectively according to 



6f p) (<Ti) oc exp 



\ct9i 

6^) (<x Q ) oc 5 ( JJ +1 j exp ( niccPi ) ,(20) 

^ = ^ + e tanh_l n tanh ^-^ ' < 21 ) 

/39i ye/3 / 

Eq. ( I21> represents a traditional form of the BP equations 
for LDPC codes [6], [7]. 

See [2] for more details of the derivations sketched in this 
Subsection. 




4$ 



Fig. 1 . Example of a general vertex model. Fourteen possible marked paths 
(generalized loops) for the example are shown in bold at the bottom. 



C. Calculating a-posteriori log-likelihoods 

Formally a-posteriori log-likelihood (magnetization) is de- 
fined by 

J2a ab W(a). (22) 



mab 



Full magnetization at an edge of a general graphical model 
can be recalculated using Eqs. dl7l22t with the gauge fields 
fixed according to the BP rules il It . There are two comple- 
mentary ways to perform these calculations. One can incor- 
porate infinitesimal test-field, h a b, into the factor functions 
by fa — ► fa Gxp(a a bh a b), and use the generalized partition 
function to generate the corresponding magnetizations by 
simply differentiating InZ (with Z taken in the loop-series 
representation) with respect to the test-field followed by 
taking the h a b — > limit. Alternatively and straightforwardly, 
one can calculate the magnetization according to the defini- 
tion (12 2 1 with the probability measure taken according to 
Eq. (|8} in the following BP-gauge form 



v bc 



be. 



2 cosh (rib c +r] c b) 



(23) 



Applying the same "marked path" rule to calculate the 
magnetization as we used in the previous Subsection to 
derive the loop-series expression for the partition function, 
we arrive at 



Abp) 



a(fC 



Abp) 



m ab 



5i 



Abp) 
l a^b;C 



1+ EKC) + E *"C* 6i0 , 

c / c a ^ b 

l + £r(C) 

c 

(bp) ff (b P ) 

c£C 



A b P)\2\ 
ib ) ) 



n (i-K 

(ab)eC 

J2 b< a bP \(Ta)cr a b II { a ac 
(bp) _ "a cEa,C 
Ma->6:C — 



ma b c P) ) 



1 



(24) 



(25) 



(26) 



where C a ^b consists of an extended family of loops with 
the connectivity degree at node a being one or higher while 
the connectivity degree at node b, as well as at any other 
node that belongs to C a ^b and different from a, being two 
or higher. 

Note that for any generalized loop C there may be many 
extended loops C a b propagating correlations imposed by the 
local loop contribution all over the graph. 

Replacing the hole families of the generalized loops C and 
the extended generalized loops C a ^,b in Eq. d25t by some 
relevant subfamilies constitutes an approximation which pro- 
vides an improvement over the bare BP approximation. 

D. Bethe-Free energy and Linear Programming Decoding 

The variational approximation for a general vertex model 
reads as follows [13]. The Bethe free energy 



(ab) a ab 



bab In b ab , 



(27) 



is a functional of the beliefs b a (cr a ), b ac (a ac ), defined on the 
vertices and edges of the graph respectively. BP equations 
can be introduced as equations for a conditional extremum 
of the Bethe-free energy. The realizability conditions (con- 
straints) are 

Va,c;c<Ea: < b a (a a ), b ac (a ac ) < 1, (28) 
V a, c; c e a : ^ b a(°~a) = ^ b °<c(cr ac ) = 1, (29) 

bac{Oac)= ^b a ((T a )= ^b c ((T c ), (30) 

&a\&ac (T c \ff ac 

where we assume a ac = a ca . The second term on the rhs 
of Eq. d27t is the entropy term that corrects for "double 
counting" of the link contribution: any link appears twice in 
the entropy part of the first term on the rhs of Eq. (I27> . 

Optimal configurations of beliefs minimize the Bethe free 
energy J27> subject to the constraints (128129130b . Introduc- 
ing the constraints as Lagrange multipliers in the effective 
Lagrangian and looking for the extremum with respect to all 
possible beliefs leads to 

« /.k) n ex p(^ )<7 <* b )' (3i) 

66a 

^Mocexp^f+^V^), 02) 

where oc indicates that one should use the normalization 
conditions d29i to guarantee that the beliefs sum up to one. 
It is straightforward to check that the system of Eqs. (13 1 132i 
supplemented with the normalization and consistency condi- 
tions for the beliefs Eqs. (1291301 is fully equivalent to the 
BP equations (1 1 1 i discussed above. 

If the aforementioned optimization procedure is performed 
with the compatibility conditions d30t excluded, yet all other 
constraints accounted for, one still obtains Eqs. (13 1 132i for 
the beliefs with rf bp > replaced by yet unconditioned 77. 
Expressing the beliefs in terms of the 7/-fields according to 
the relaxed version of Eqs. (I3 H32i we arrive at the following 
expression for the Bethe free energy in terms of the 77 
variables: 

{ab) 

J2 a abfa((ra) l\ exp(?7 ac Cr QC ) 

ro W = £2 c ll (vx\ 

a ^ b ~ E/aK) I! ex P (j7 ac C7 ac ) ' ^ 
m ab = tanh iVab + Vba), (34) 

where J-o(rf) = — hiZ Q (ri) and Tn a *}^ b and rrv*) are two 
expressions for a-posteriori log-likelihoods (magnetizations) 
at the edge (ab) which are equal to each other if the compati- 
bility conditions are accounted for. For an arbitrary choice of 
77 the magnetizations are different. However, for the special 
choice of 77 that corresponds to r)( bp > the two generally 
different magnetizations become equal. More formally: the 
Belief Propagation equations can be expressed as 

"£l 6 l(*)-(6i0 = ™i*i a l(*)-(M = m ab\(*)-+{b P ), (35) 



satisfied Va&, where 77 and M** 1 turn into 77^) and b^ bp \ 
respectively. 

There is also a relation between the Bethe free energy 
minimization approach and the LP decoding [30], [21], [35], 
[36]. Consider the Bethe free energy Eq. ( I27> in the / — > 00 
limit of zero noise or, equivalently, large SNR. In this limit 
the self-energy contribution to the free energy dominates 
the entropy terms and the latter can be safely neglected. 
However, both the self energy and the constrains d28l29l30l > 
are linear in beliefs. Therefore this asymptotic optimization 
problem can be solved efficiently by means of the standard 
linear programming approach. 

II. Loop calculus analysis of the LP-instantons 

FOR THE (155, 64, 20) CODE 

We consider a family of (~ 200) instantons, i.e. noise 
configurations (log-likelihoods) corresponding to the pseudo- 
codewords with the effective distance smaller then the 
Hamming distance of the code. This set of instantons and 
related pseudo-codewords was found in [31] for the Tan- 
ner (155, 64, 20) code performing over the Additive-White- 
Gaussian-Noise (AWGN) channel and decoded using LP 
decoding. 

In short the method/algorithm of [31], called pseudo- 
codeword search algorithm, proposes an efficient way of 
describing LP decoding polytope and the pseudo-codeword 
spectra of the code. It approximates a pseudo-codeword and 
the corresponding noise configuration on the error-surface 
surrounding the zero codeword correspondent to the shortest 
effective distance of the code. The algorithm starts from 
choosing a random initial noise configuration. The config- 
uration is modified through a discrete number of steps. Each 
step consists of two sub-steps. First, one applies LP decoder 
to the initial noise-configuration deriving a pseudo-codeword. 
Second, one finds the noise configuration equidistant from 
both the pseudo codeword and the zero codeword. The 
resulting noise configuration is used as an entry for the next 
step. The algorithm, tested on the Tanner (155,64,20) code 
and Margulis p — 7 and p = 11 codes (672 and 2640 bits 
long respectively) shows very fast convergence. 

Discussing the instantons, found for the (155, 64, 20) 
code with the pseudo-codeword-search algorithm, one after 
another we aim to associate for each instanton the corre- 
sponding critical loop, T that generates a contribution to the 
loop series dl8> . comparable to the bare BP contribution. 

We restrict our search for the critical loop contribution to 
the class of single-connected loops, i.e. loops that consist of 
checks and bits with each check connected to only two bits 
of the loop. According to Eqs. i ll 8b such contribution to the 
loop series is the product of all the triads, l^ hp \ along the 
loop, 

r(T)=l[^, (36) 

{bp) 

jj<M = ^2 (37) 

^(i-( m ^Y)(i-( m ^y) 



where for any check a that belongs to T, i, j is the only pair 
of a bit neighbors that also belongs to T. By construction, 
lAenyl — ^ e immediately find that for the critical 
loop contribution to be exactly equal to unity (where unity 
corresponds to the bare BP term), the critical loop should 
consist of triads with all equal to unity by absolute 

value. Even if degeneracy is not exact one still anticipates the 
contributions from all the triads along the critical loop to be 
reasonably large, as an emergence of a single triad with small 
ffip) will make the entire product negligible in comparison 
with the bare BP term. This consideration suggests that an 
efficient way to find a single connected critical loop, T, with 
large |r(T)| consists of, first, ignoring all the triads with 
|/x( bp ) | below a certain 0(1) threshold, say 0.999, and second 
checking if one can construct a single connected loop out of 
the remaining triads. If no critical loop is found we lower 
the threshold till a leading critical loop emerges. 

Applied to the set of instantons of the Tanner (155, 64, 20) 
code with the lowest effective distances this triad-based 
scheme generates r(T) that is exactly unity by the absolute 
value. This is the special degenerate case when the critical 
loop contribution and the BP/LP contribution are equal to 
each other by absolute value. Thus only the sixth of the 
first dozen of instantons has r(T) « 0.82 while all others 
yield r(T) = 1. To extend the triad-based search scheme 
to the instantons with larger effective distance one needs to 
decrease the threshold. This always results in emergence of 
at least one single connected loop with r(T) ~ 1. Note, 
that it may be advantageous, even thought not necessary, 
to include in this triad-based search for the critical loop 
some additional criteria. In particular, we found it useful to 
also require that the absolute values of the a-posteriori log- 
likelihoods of the bits involved in the critical loop be larger 
than certain threshold. 

Fig. |2] shows a representative set of instantons analyzed 
using the described loop calculus tool. Critical loops are 
shown in the corresponding subplots. The resulting critical 
loops are typically 4 — 5 bits long (with the girth of the 
code that counts both bits and checks being 8). However 
some configurations, like instanton #192 shown in the right 
lower corner of Fig. (|2ji, corresponds to a highly degenerate 
situation. The instanton #192 shows three distinct single 
connected loops giving all r(T) = 1 contribution to the loop 
series Jl 81 . 

Obviously, when an instanton produces a critical loop T 
with r(r) = 1 one can guarantee that this is the largest 
non-bare (non-BP) contribution to the loop series. However 
in all other cases when r(T) < 1, no guarantee can be 
given and we may not exclude a possibility that some other 
general loop of a more complex structure can provide a 
contribution to the loop series with higher r(T). However, 
we will show in Section ffVl that knowledge of the r(T) — 
0(1) critical contribution found along the way explained 
above is sufficient for successful decoding of these dangerous 
configurations. 

One final remark of this Section concerns the value of 
magnetization calculated for degenerate critical loops, i.e. 



loops with r(T) = 1. Calculating a-posteriori log-likelihood 
(magnetization) at a bit which belongs to the critical loop, 
one finds that the first term in the numerator of Eq. P4I 
is completely compensated by the only relevant of the 
other terms, correspondent to T a ^i, replaced just by the 
critical loop V. Therefore, if only these two contributions 
are accounted for the magnetization at the bit is exactly 
zero. This suggests that one of the effects related to a critical 
loop is an effective shift of log-likelihoods at the bits of the 
critical loop in the direction opposite to the magnetization 
measured at the bit by bare BP. We have also found that 
the bare BP a-posteriori-log-likelihoods along the critical 
loop are always aligned bit-wise with the corresponding log- 
likelihoods. This set of observations will actually be explored 
in Section ffVl to construct a simple modification of the LP 
decoding algorithm. 



III. Effective Free Energy Approach 

Accounting for a single loop effect, when it is comparable 
to a bare (BP) contribution, can be improved through an 
effective free energy approach explained in this Section. 
This approach is akin to degenerate Hartree-Fock variational 
approach used in quantum mechanics (quantum chemistry) 
[37] in the case of a phase space degeneracy. 

It was shown in the previous Section that finding the 
BP gauge is equivalent to optimizing (finding an extremum 
of) the functional J-o(rj) = — IuZq, where Zq is given by 
Eq. J13i . Therefore, the 77-gauge is fixed according to the 
first term in the series Eq. (|8jl. Further terms in Z, coming 
from other higher-order vertex "corrections", were calculated 
above with the BP gauge fixed. This resulted in Eqs. 11711 8i . 

As we see from an example presented in the previous 
Section, some number (or just one) of these corrections 
may be either comparable (by absolute value) or just equal 
to the bare Zq contribution. Any of this special O(Zq) 
contributions is associated with some critical generalized 
loop. This situation, when one or more of these critical loops 
emerge, is a potentially dangerous one, possibly leading (as 
we observed) to the bare BP failure. 

In this troublesome situation a plausible solution would be 
to modify the BP gauge conditions Eq. ill 2b by 
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where Zr is the component of the full expression for Z 
that corresponds to a critical loop, T. In gener al Eq. 01 
will be different from the standard BP equation, and we can 
anticipates that the corrections due to the critical loops may 
cure the bare BP failure in decoding. 

From Eqs. J38I we derive the following set of modified 



BP equations 
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where Eas. (l33l34> defined m*^(rf) and rn ( *_^ b (r]) and 
Eqs. d39l should all be taken at rj — » r; c ff as the system of 
equations actually defines r/efr- Recast in terms of the beliefs 
Eqs. d39l forms a system of polynomial equations for beliefs, 
that become linear only if the right hand sides of Eqs. (1391 
turn to zero. 

To find a-posteriori log-likelihoods within the degenerate 
approach one calculates 
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where r\ is substituted by rj c g solving Eqs. i39i . 

Note a couple of special cases. First of all, if the graph 
consists of a set of disconnected single loops, the modified 
BP equations j39t are simply reduced to the standard BP, 
with the rhs of Eq. d39l replaced by zero. One consequence 
of this degeneracy is that if one chooses T based on all the 
single-connected loops contained in the degenerate model, 
the variational result would just be exact. Second, if one 
considers contributions to T based on some number of 
single-connected critical loops, T, the only terms on the rhs 
of Eqs. ( I39t that will not result in exact zero for the bare BP 
solution will be associated with a — > b where a G T while 

b £ r. 

The modified free energy approach, described by equations 
d39l for renormalized gauges ?i e fj, promises decoding benefit 
in the degenerate or close to the degenerate cases when com- 
pared with the corresponding direct truncation of the loop 
series given by Eqs. d!7l > or Eqs. dl 81 . The approximation is 
also convenient as it keeps the same level of complexity as 
the BP equations. This is in contrary to the direct approach 
which requires summation according to Eq. d25t over many 
extended diagrams for getting renormalized values of log- 
likelihoods at the bits that do not belong to the critical loops. 

Note that pretty much like in the case of bare BP, to define 
an algorithm associated with Eqs. \39\ one needs to introduce 
an iterative algorithm based on it, and there is obviously 
some freedom associated with the choice of discretization. 
(See [38] for discussion of different discretization/iteration 
schemes in the context of the bare BP equation.) 

We expect that accounting for just one critical loop V that 
corresponds to the largest value of rr (calculated within 



the bare BP) will be already sufficient for a substantial 
improvement of BP in the special cases when the bare BP 
fails. Summarizing, we arrive at the following 
Loop-corrected BP algorithm 

« 1. Run bare BP algorithm. Terminate if BP succeeds 

(i.e. a valid code word is found in terms of marginal 

probabilities). 

• 2. If BP fails find the most relevant loop Y that cor- 
responds to the maximal (by absolute value) amplitude 
rr in Eq. d!7l . A simple method for the critical loop 
search introduced in the previous Section may be tried 
first. 

• 3. Solve the modified-BP equations d39i for the given 
T. Terminate if the improved-BP succeeds. 

« 4. Return to Step 2 with an improved T-loop selection. 
An additional loop found through an improved critical 
loop procedure can simply be added to the sum on the 
rhs of Eqs. d39l . 
In this manuscript we do not report any results of nu- 
merical simulations where the Loop-corrected BP algorithm 
would be tested directly using a sample code. We postpone 
this important exercise for future analysis. Instead, we use 
the result described in this Section as a motivation for an 
even simpler heuristic approach detailed in the next Section. 

IV. LP-ERASURE DECODING 

As it was already discussed in the literature [35], [31] and 
commented on in Section II-DI LP decoding can be viewed 
as a certain (large SNR) limit of BP decoding. It is not 
obvious, however, that the BP-improved procedure outlined 
in the previous Section can be rigorously transformed into a 
correction to LP keeping a linear structure. 

Our approach to this question will be heuristic, i.e. we 
simply conjecture a plausible modification of the LP scheme 
based on the algorithm formulated above for improved BP 
and then test this idea using an example of the (155, 64, 20) 
code. 

On our way to proposing an improved LP decoding we 
first note that the effective free energy approach keeps the 
same number of degrees of freedom as the original bare 
BP. Therefore, if we conjecture that a modification of LP 
decoding should keep its linear structure and thus the number 
of constraints, the only actual degree of freedom left is 
in the log-likelihoods, i.e. in their possible modifications 
deduced from loop calculus. We further observe that the 
modifications of the BP equations discussed in the previous 
Section are actually well localized. Specifically, the rhs of 
d39l is non-zero for the 77 variables associated with the 
vertices that belong to the critical loop V or are immediately 
adjusted to it. Given that LP decoding is a special limit 
of BP decoding one deduces from these observations that 
the log-likelihoods should be renormalized just at the bits 
lying on the critical loop. Furthermore, taking into account 
the observation reported in the last paragraph of Section |n] 
we argue that renormalization of log-likelihoods on the bits 
of the critical loop should be directed against the bare log- 
likelihoods. 



All this suggests the following LP-version of the loop- 
enhanced algorithm: 
LP-erasure algorithm 

• 1. Run LP algorithm. Terminate if LP succeeds (i.e. a 
valid code word is found). 

• 2. If LP fails, find the most relevant loop T that 
corresponds to the maximal amplitude r(T) in the LP- 
version of Eq. H7\ . 

• 3. Modify the log-likelihoods (factor-functions) along 
the loop r introducing a shift towards zero, i.e. intro- 
duce a complete or partial erasure of the log-likelihoods 
at the bits. Run LP with modified log-likelihoods. 
Terminate if the modified LP succeeds. 

• 4. Return to Step 2 with an improved selection principle 
for the critical loop. 

Let us also mention that our loop-calculus based conjec- 
ture that the full or partial erasure of certain log-likelihoods 
is beneficial for LP decoding is akin to the statement made 
in [39] in regard to the positive effect of thresholding in LP 
decoding. 

A. (155, 64, 20) test of the LP-erasure algorithm 

Here we describe our numerical test of the LP-erasure 
algorithm. The test is based on the analysis (described 
in Section HJ of instantons, i.e. most probable out of a 
variety of dangerous noise configurations that lead to LP- 
decoding failure. The ~ 200 instantons found in [31] for 
the 155, 64, 20) code have the effective weight lower then 
the Hamming distance of the code, thus leading to the 
undesirable error-floor. We apply the triad method of Section 
ITT1 to analyze all the low effective weight instantons of the 
(155, 64, 20) code. In spite of the fact that the method did 
not guarantee that the critical loop was actually the one with 
the largest (for given configuration of the noise) r(T) we 
still choose to try it for the next, third, step of the LP- 
erasure procedure: for the special marked bits we lowered 
the original log-likelihood uniformly multiplying all the log- 
likelihoods at the marked bits of the critical loop by a positive 
number e that is smaller than one. 

The results of the test are remarkable. We found out that all 
instantons are actually corrected already with the roughest, 
i.e. e = 0, modification corresponding to the full erasure 
of the information (log-likelihoods) along the critical loop. 
We verified that the noise configurations that are re-scaled 
instantons (of the same structure but with effective distance 
larger then one of the original instanton but smaller then the 
Hamming distance of the code) are also corrected by the 
LP-erasure algorithm successfully. 

Note that the instantons shown in Fig. (|2jl are counted 
using the all "+1" configuration as the original codeword 
primarily for the purpose of the demonstration transparency. 
We did verify that the LP-erasure algorithm is invariant with 
respect to a change in the original codeword, i.e. that the LP- 
erasure algorithm corrects instantons of the bare LP and their 
derived configurations if counted using any other codeword 
as a reference point. In all our tests (with the ~ 200 



instantons) whenever LP-erasure decoded to a codeword, the 
codeword was actually the right one. 

The LP-erasure algorithm also shows an impressive ro- 
bustness. The algorithm often forgives an inaccurate defini- 
tion of the critical loop. For example, if one uses a very 
low threshold in identifying the bits that can possibly enter 
the critical loop, the resulting loop can actually be large and 
contain up to 20 bits. By lowering or completely erasing log- 
likelihoods at all these bits we often get the correct result 
with subsequent LP-decoding. However, in the rare cases 
when this loose way to define a critical loop leads to a failure 
one just needs to tighten the threshold and possibly use some 
additional thresholding (e.g. in the value of the a-posteriori- 
loglikelihood that belong to the loop and also in the erasure 
coefficient e) till a codeword emerges. 

V. Conclusions and Discussions 

In this manuscript we presented a proof-of-concept test 
demonstrating the utility of the loop calculus approach of 
[1], [2] to improve inference algorithms of the BP class in 
general, and of BP decoding of LDPC codes in particular. 
The key observation, that enabled the reported improvement 
in the decoding scheme, was emergence of a well-defined and 
relatively simple loop correction for each BP-dangerous con- 
figuration of log-likelihoods. Identification of the critical loop 
and further log-likelihood specific modification of the BP/LP 
algorithm has been suggested as a cure for bare BP/LP 
failure. LP-erasure that is the simplest algorithm based on 
the critical loop identification, was successfully tested using 
the (155, 64, 20) code operated on the AWGN channel. LP- 
erasure was able to correct all bare-LP-dangerous noise con- 
figurations related to the previously found pseudo-codewords 
with the effective distance lower than the Hamming distance 
of the code. 

This demonstration is clearly the first step in the high- 
lighted direction where the next steps are thought of as 
follows. We plan to improve and continue testing the simple 
LP-erasure algorithm. The major improvement required is 
an automatization of the critical loop identification scheme. 
Further tests imply (a) direct comparison of LP-erasure with 
bare LP algorithms with the help of Monte Carlo simulations 
of BER/FER, (b) working with other (longer) codes, (c) 
working with other (e.g. correlated) channels. We will also 
be implementing all of the above using a more sophisticated 
and also better justified Loop-corrected BP scheme outlined 
in Section ITTTl 

These studies will certainly benefit from using some other 
recent developments in the field of BP/LP decoding, such 
as [40], [41] on reducing complexity of LP-decoding, [38] 
on accelerating the bare BP convergence , and [42] which 
suggests an alternative method of LP-decoding improvement. 
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Fig. 2. The panels represent the results of LP decoding and critical loop identification for 6 representative instantons found for the Tanner (155, 64, 20) 
code performing over AWGN channel. All instantons has an effective distance smaller than the Hamming distance, 20, of the code and the exact (ML) 
decoding would correctly decode them to the correct codeword (all +1 for the test shown in the Figure). The main Figures show dependence of the 
a-posteriori log-likelihood on the bit label/position. Bits lying on the critical loops are marked with red (filled circles). The critical loops are also shown 
schematically in the subplots. Values shown in the sub plot s next to the checks (squares) that connect pairs of bits on the critical loop are related to the of 
the corresponding triad contributions jl defined by Eq. 1371 . Loop-erasure decoding, with erasures applied along the critical loop, corrects all the dangerous 
(instanton) errors. 
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